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In this issue, Lomuscio and Penczek survey some of the recent work in verification of temporal- 
epistemic logic via symbolic model checking, focussing on OBDD-based and SAT-based approaches 
for epistemic logics built on discrete and real-time branching time temporal logics. 

On this topic, I should mention the following paper, which compares several model checkers for 
epistemic logics with a temporal component, using as a test case the Russian Cards problem: 

H. P. van Ditmarsch, W. van der Hoek, R. van der Meyden, and J. Ruan. Model 
checking Russian Cards. Electronic Notes in Theoretical Computer Science, 149(2): 105- 
123, 2006. 

The Russian Card problem is described here: 

H. P. van Ditmarsch. The Russian Cards problem. Studia Logica, 75:31-62, 2003. 



Symbolic Model Checking for Temporal-Epistemic 



1 Introduction 

The study of epistemic logics, or logics for the representation of knowledge, has a long and successful 
tradition in Logic, Computer Science, Economics and Philosophy. Its main motivational thrust is 
the observation that knowledge of the principals (or agents) in an exchange is fundamental in the 
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study not only of the information they have at their disposal, but also in the analysis of their 
rational actions and, consequently, of the overall behaviour of the system. It is often remarked 
that the first systematic attempts to develop modal formalisms for knowledge date back to the 
sixties and seventies and in particular to the works of Hintikka [28] and Gettier [37]. The line of 
work at the time focussed on the adequacy of particular principles, expressed as axioms of modal 
logic, representing certain properties of knowledge in a rational setting. The standard framework 
consisted of the propositional normal modal logic S5 n [10] built on top of the propositional calculus 
by considering the axioms 
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together with usual normal rules of necessitation Nec : From tp infer Kiip and modus ponens. Since 
then several other formalisms have been introduced accounting for weaker notions of knowledge as 
well as subtly different mental notions such as belief, explicit knowledge and others. 

While in the sixties soundness and completeness of these formalisms were shown, the standard 
semantics considered was the one of plain Kripke models. These are models of the form M = 
(W, {Ri}i£A, V), where W is a set of "possible worlds", Ri C W x W is a binary relation between 
worlds expressing epistemic indistinguishably between them, and V : W — > 2 PV is an interpretation 
function for a set of basic propositional variables PV. Indeed, much of the theory of modal logic has 
been developed in this setting up to recent times. However, in the eighties and nineties attention 
was given to finer grained semantics that accounted for the particular states of computation in a 
system. In terms of epistemic logic the challenge was to develop semantics that accounted both to 
the low-level models of (a-) synchronous actions and protocols, and that at the same time would 
be amenable to simple yet intuitive notions of knowledge. The key basic semantical concept put 
forward at the time satisfying these considerations was the one which became popular with the 
name of interpreted system. Originally developed independently by Parikh and Ramanujam [50] . 
Halpern and Moses [26] and Rosenscheim |61| and later popularised by [22] , the interpreted system 
model offered a natural yet powerful formalism to represent the temporal evolution of a system as 
well as the evolution of knowledge of the principals in the run. The development of this model 
triggered a tremendous acceleration in the study of logics for knowledge with several results being 
produced both in terms of axiomatisations with respect to several different classes of models of 
agents (synchronous, asynchronous, perfect recall, no learning, etc.) as well as applications of these 
to standard problems such as coordinated attack, communication, security, and others. 

In this setting logic was most often seen as a formal reasoning tool. Attention was given to the 
exploration of metaproperties of the various formalisms (such as their completeness, decidability, 
and computational complexity), axiomatisations developed. Attempts were made to verify systems 
automatically by exploring the relation T \~l <p, where (p is a specification for the system, L is the 
axiomatised logic representing the system and T, a set of formulae expressing the initial conditions. 
However, partly due to the inherent complexity of some of the epistemic formalisms, verification of 
concrete systems via theorem proving for epistemic logics did not attract too much attention. 

At the same time (the early nineties) the area of verification by model checking [17] began 
acquiring considerable attention with a stream of results being produced for a variety of temporal 
logics. The idea of switching attention from theorem proving to model checking became prominent 
|27j . However, it was not before the very end of the nineties that similar ideas began becoming 
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applied to the verification of multi-agent systems via temporal-epistemic formalisms. The first 
contribution in the area to our knowledge dates back to a paper by van der Meyden and Shilov 
|47j . where the complexity of model checking perfect recall semantics is analysed. After that 
attention switched to the possible use of ad-hoc local propositions for translating the verification 
of temporal-epistemic into plain temporal logic [29] . Following this there were studies on the 
extension of bounded model checking algorithms [53] and binary-decision diagrams [60J. Several 
other extensions and algorithms later appeared. 

The aim of this paper is to survey some of the results by the authors in this area over the past 
few years. The area has grown tremendously and it is impossible to provide a comprehensive yet 
technical enough survey in a relatively compact article; some other approaches are discussed in 
Section [U but others, inevitably, are unfortunately left out. In particular here we only consider 
approaches where knowledge is treated as a full-fledged modality interpreted on sets of global states 
in possible executions and not as a simple predicate as other approaches have featured. Concretely, 
the rest of the paper is organised as follows. In Section [2] we present syntax and semantics of 
the basic logic. In Section [3] we introduce and discuss an OBDD-based approach to verification 
of temporal-epistemic logic. In Section 0] an alternative yet complementary approach based on 
bounded and unbounded model checking is discussed. In Section [5] extensions to real-time are 
summarised briefly. Related work is discussed in Section [6J 

2 Syntax and Semantics 

Many model checking approaches differ depending on the syntax supported as a specification lan- 
guage for the properties to be verified by the model checker. We begin here with the basic temporal 
branching time temporal-epistemic logic. 

2.1 Syntax 

Given a set of agents A = {1, . . . , n} and a set of propositional variables PV, we define the language 
C of CTLK as the fusion between the branching time logic CTL and the epistemic logic S5 n for n 
modalities of knowledge Ki {i = 1, . . . , n) and group epistemic modalities Ep, Dp, and Cp (r C A): 

ip, ip ::=p £ PV | -xp | (p A ip | Knp \ Epp \ Dpp \ Cp(p \ AXip \ AGp \ A(<pTJip) 

In addition to the standard Boolean connectives the syntax above defines two fragments: an epis- 
temic and a temporal one. The epistemic part includes formulas of the form Knp representing 
"agent i knows that ip" , Epip standing for "everyone in group T knows that (p" , Dpip representing 
"it is distributed knowledge in group V that <p is true" , Cp formalising "it is common knowledge in 
group r that p" . We refer to [22] for a discussion of these concepts and examples. The temporal 
fragment defines formulas of the form AXp meaning "in all possible paths at each possible next 
step tp holds true"; AG<^ standing for "in all possible paths along p is always true"; and A(pVip) 
representing "in all possible paths at some point ip holds true and before then ip is true along the 
path" . 

Whenever T = A we will omit the subscript from the group modalities E, D, and C. As 
customary we will also use "diamond modalities", i.e., modalities dual to the ones defined. In 
particular, for the temporal part we use EFp = —iAG—>p, EX99 = —iAK—xp representing "there 
exists a path where at some point p is true" and "there exists a path in which at the next step ip 
is true" respectively. We will also use the E(ip\Jip) with obvious meaning. For the epistemic part 
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we use overlines to indicate the epistemic diamonds; in particular we use Knp as a shortcut for 
—iKi—iip, meaning "agent i considers it possible that <p" and similarly for Ep, D?, and C-p. 

Formulas including both temporal and epistemic modalities can represent expressive specifica- 
tions in particular scenarios, e.g., the evolution of private and group knowledge over time, knowledge 
about a changing environment as well as knowledge about other agents' knowledge. We refer to 
[22| for standard examples such as alternating bit protocol, attacking generals, message passing 
systems, etc. 

2.2 Interpreted systems semantics 

In what follows the syntax of the specification language supported is interpreted on the multi-agent 
semantics of interpreted systems |22| . Interpreted systems are a fine-grained semantics put forward 
in [26] to represent temporal evolution and knowledge in multi-agent systems. Although initially 
developed for linear time, given the applications of this paper we present them in their branching 
time version. Given the model checking algorithms described later we summarise the formalism 
below in relation to a branching time model. For more details we refer to |22j . 

Assume a set of possible local states Li for each agent i in a set A = {1, . . . , n} and a set L e of 
possible local states for the environment e. The set of possible global states G C L\ X • • • X L n X L e 
is the set of all possible tuples (l±, . . . , l n , l e ) representing a snapshot of the system as a whole. The 
model stipulates that each agent i performs one of the enabled actions in a given state according to 
a protocol function Pi : Li — > 2 Acti . Pi maps local states to sets of possible actions for agent i within 
a repertoire of its actions Acti. Similarly, the environment e is assumed to be performing actions 
following its protocol P e : L e — > 2 Acte . Joint actions (act\, . . . ,act n ,act e ) are tuples of actions 
performed jointly by all agents and the environment in accordance with their respective protocol. 
Joint actions are used to determine the transition function TCGx Act\ x • • • x Act n x Act e x G 
which gives the evolution of a system from an initial global state g° € G. A path ir = (go,gi, • • •) is 
a maximal sequence of global states such that (gk,9k+i) S T for each k > (if tt is finite then the 
range of k is restricted accordingly). For a path ir = {go,gi, ■ ■ ■), we take ir(k) = gu- By H(g) we 
denote the set of all the paths starting at g € G. 

The model above can be enriched in several ways by expressing explicitly observation functions 
for the agents in the system or by taking more concrete definitions of the sets of local states 
thereby modelling specific classes of systems (perfect recall, no learning, etc.). We do not discuss 
these options here; we simply note that in a later section we will pair this semantics with an 
automata-based one. 

To interpret the formulas of the language C for convenience we define models simply as tuples 
M = (G, g°, T, ~Xj • • ■ where G is the set of the global states reachable from the initial 

global state g° via T; ~j C G x G is an epistemic relation for agent i defined by g ~j g' iff 
h(g) = k(g'), where k : G — ► Lj returns the local state of agent i given a global state; and 
V : G x PV — > {true, false} is an interpretation for the propositional variables PV in the 
language. 

The intuition behind the definition of models above is that the global states whose local com- 
ponents are the same for agent i are not distinguishable for the agent in question. This definition 
is standard in epistemic logics via interpreted systems — again we refer to [22j for more details. 

We can use the model above to give a satisfaction relation |= for C inductively as standard. Let 
M be a model, g = (l±, . . . , l n ) a global state, and <p, ip formulas in L: 

• (M, g)\=piS V(g, p) = true, 

• (M, g) h Kiip iff for all g' £ G if g ~ 4 g' , then (M, g>) \= <p, 
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• (M, g) (= D V ip iff for all i 6 T and g' G G if g ~* then (M, g') |= (/?, 
. (M, 5 ) h ^ iff (M, 5 ) h KerKi^P, 

• (M,g) |= C r ^ iff for all k > we have (M,g) |= E$tp, 

• (M, 5) |= AX(^ iff for all vr G H(flr) we have (M, vr(l)) |= 99, 

• (Af, g) \= AGtp iff for all vr G 1%) and for all k > we have (M, vr(/c)) |= 92, 

• (M,5f) |= A(^U^) iff for all 7r G 11(5)) there exists afc>0 such that (M,7r(fc)) |= V and for 
all < j < k we have (M, ir(j)) \= <p. 

The definitions for the Boolean connectives and the other inherited modalities are given as standard 
and not repeated here. E k ip is to be understood as a shortcut for k occurrences of the E modality 
followed by 99, i.e., E°ip = ip; E l ip = Etp; E k+1 ip = EE k ip. 

2.3 The dining cryptographers problem 

The formalism of interpreted systems has been used successfully to model a variety of scenarios 
ranging from basic communication protocols (e.g., the bit transmission problem, message passing 
systems), to coordination (e.g., the attacking generals setting), deadlocks (e.g., the train-gate- 
controller scenario), etc. We refer the reader to the specialised literature; the key consideration 
here is that in each of these scenarios it is shown that temporal-epistemic languages can be used 
to express specification for the systems and the individual agents very naturally. 

To exemplify this we present a protocol for anonymous broadcast very well-known in the se- 
curity literature: the dining cryptographers (DC). The DC was introduced by Chaum [14] and 
analysed in a temporal-epistemic setting by Meyden and Su [H]. A reformulation to include cheat- 
ing cryptographers (see Section [6]) appears in [33] . We report the original wording here [H] (part 
of this text was originally cited in [48]). 

Three cryptographers are sitting down to dinner at their favorite three-star restaurant. 
Their waiter informs them that arrangements have been made with the maitre d 'hotel for 
the bill to be paid anonymously. One of the cryptographers might be paying for dinner, 
or it might have been NSA (U.S. National Security Agency). The three cryptographers 
respect each other's right to make an anonymous payment, but they wonder if NSA is 
paying. They resolve their uncertainty fairly by carrying out the following protocol: 

Each cryptographer flips an unbiased coin behind his menu, between him and the cryp- 
tographer on his right, so that only the two of them can see the outcome. Each cryptog- 
rapher then states aloud whether the two coins he can see-the one he flipped and the one 
his left-hand neighbor flipped-fell on the same side or on different sides. If one of the 
cryptographers is the payer, he states the opposite of what he sees. An odd number of 
differences uttered at the table indicates that a cryptographer is paying; an even number 
indicates that NSA is paying (assuming that dinner was paid for only once). Yet if a 
cryptographer is paying, neither of the other two learns anything from the utterances 
about which cryptographer it is. 

Temporal-epistemic logic can be used to analyse the specification of the example — we summarise 
here the description reported in [60} [56] . It is relatively straightforward to model the protocol above 
by means of interpreted systems. For each agent i we can consider a local state consisting of the 
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triple (lj,lf,lf), representing respectively whether the coins observed are the same or different, 
whether agent i paid for the bill, and whether the announcements have an even or odd parity. A 
local state for the environment can be taken as a 4-tuple (l\, 1^,1^, l*) where l\, l^, l\ represent the 
coin tosses for each agent and l\ represents whether or not the agent in question paid for the bill. 
Actions and protocols for the agents and the environment can easily be given following Chaum's 
narrative description above and relations for the temporal evolution and the epistemic relation 
easily built in this way. 

In principle by coding the above we would be able to show on the model for DC that 

(M DC ,g°) \= /\ (odd A -paid;) AX(i^(\/ paidj) /\ ^paid k ) 

i£A j^i k^i 

The specification above states that if an agent i observes an odd parity and did not cover the bill 
then in all next states (i.e., when the announcements have been made) she will know that one of 
the others paid for dinner but without knowing who it was. 

Although conceptually easy, the example is already large enough to make it difficult to work 
out all possible execution traces on the model. Of note is the fact that DC can actually be scaled 
to any number of cryptographers. By using model checking techniques one can verify DC up to 
8 and more cryptographers with resulting state spaces for the model of about 10 36 states, and 
considerably more cryptographers if the representation of the model is optimised |33j. 

Other examples are equally amenable to representation via interpreted systems and model 
checking via the techniques presented below. 

3 OBDD-based symbolic model checking 

As it is customary in model checking in the following we analyse systems of finite states only. Given 
a system S and a property P to be checked, the model checking approach suggests coding S as a 
logical model Ms, the property P as a logic formula (pp, and investigating whether Ms \= pp. In the 
traditional approach the model Ms is finite and represents all the possible computations of system 
S and ipp is a formula in temporal logic expressing some property to be checked on the system, 
e.g., liveness, safety, etc. When (pp is given in LTL or CTL checking ipp on an explicitly given 
Ms is, of course, a very tractable problem. However it is impractical to represent Ms explicitly, so 
Ms is normally implicitly given by means of a dedicated programming language using imperative 
commands on sets of variables. This can be convenient for the programmer, but the number of 
states in the resulting model grows exponentially with the number of variables used in the program 
describing Ms potentially causing great difficulty (state explosion problem). 

Much of the model checking literature in plain temporal logic deals with techniques to limit the 
impact of this, the most prominent being partial order reductions [521125]. symmetry reductions |16[ 
EQjEl], ordered-binary decision diagrams [121 [45], bounded and unbounded model checking [9l [46]. 
and (predicate) abstraction [181 E] • By using partial-order reduction techniques the computational 
tree Ms is pruned and certain provably redundant states eliminated and/or collapsed with others 
depending on the formula to be checked thereby reducing the state space. Symmetry reductions 
are used to reducing the state spaces of distributed systems composed of many similar processes. 
Predicate abstraction is based on the identification of certain predicates which have no impact on 
the verification of the formula in question; crucially it is used in verification of infinite-state systems. 
Binary-decision diagrams (described below) offer a compact representation for Boolean formulas 
and traditionally constitute one of the leading symbolic approaches. Bounded and unbounded 
model checking (described in Subsections 14.11 and 14.21 respectively) exploit recent advances in the 
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efficiency of checking satisfiability for appropriate Boolean formulas suitably constructed. Several 
tools have been developed for model checking temporal logic, including SPIN [30J for partial-order 
reductions for LTL, SMV and NuSMV [45 [ [To] for binary-decision diagrams and bounded model 
checking for LTL, and SLAM [7] for partial-ordered reductions for safety properties. Several other 
tools exist for other varieties of temporal logic, e.g., real-time logics, probabilistic temporal logic, 
and indeed other implementations are available for the same or slightly different techniques. 

Even if all tools mentioned above are nowadays very sophisticated and support ad-hoc input 
languages they are limited to temporal logics only. In the rest of the paper we discuss techniques 
and tools supporting temporal-epistemic logics. 

3.1 The ordered binary decision diagrams approach 

The two main model checking platforms for temporal-epistemic logic based on binary-decision 
diagrams are the MCK and the MCMAS toolkits. Both in their experimental phase, they implement 
model checking of temporal-epistemic logic on interpreted systems semantics via ordered-binary 
decision diagrams. MCK [231 [JJ implements a variety of different semantics (observational, perfect 
recall, etc), supports a concise and specialised input language, and was the first model checker 
available supporting temporal-epistemic logic. MCMAS |57[ HQ] implements standard interpreted 
systems semantics and a number of extensions, including deontic modalities, explicit knowledge, 
ATL, etc. In terms of implementations the two tools are rather different. MCK is implemented 
in Haskell using Long's BDD library (written in C), whereas MCMAS is implemented in C++ 
and relies on Somenzi's [62] BDD package (also in C). MCMAS and its theoretical background is 
succinctly described in the rest of this section; we refer to [56] for an in-depth description. 

Irrespective of the implementation details the angle when working on ordered-binary decision 
diagrams (OBDDs) is the symbolic (OBDD-based) representation of sets and functions paired with 
the observation that to work out whether (M, g) \= <p it is sufficient to evaluate whether or not 
g G SAT (tp) where SAT((p) is the set of states in the model M satisfying ip. To introduce the main 
ideas of the approach we proceed in three stages: first, we observe we can code sets as Boolean 
formulas; second, we show how OBDDs offer a compact representation to Boolean functions; third 
we give algorithms for the calculation of SAT(ip). 

First of all observe that given a set G of size |G| it is obvious how to associate uniquely a vector 
of Boolean variables (w±, . . . ,w m ) to any element g E G where m = \l0g2\G\~\. (Note that a tuple 
of m places can represent 2 m different elements). Any subset SCG can be represented by using 
a characteristic function fs : (<?i, • • • ,g m ) —> {0, 1}, expressing whether the element (as encoded) 
is in S or not. Note that functions and relations can also be encoded as Boolean functions; for 
instance to encode that two states are related by some relation we can simply consider a vector 
of Boolean functions comprising of two copies of the representation of the state to which we add 
a further Boolean variable expressing whether or not the states are related. Vectors designed in 
this way represent conjunctions of Boolean atoms or their negation and as such constitute a simple 
(albeit possibly long) Boolean formula. 

In the construction of OBDD-based model checking for plain temporal logic it is normally as- 
sumed that the propositions themselves (appropriately ordered) constitute the basis for the encoding 
of the states of the model. In the MCMAS approach Boolean functions first and then OBDDs are 
constructed iteratively by considering all aspects of the interpreted system given. These involve 
building the: 

• Boolean functions for the sets of local, global states, actions, initial global states; 
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Figure 1: A BDT for the Boolean function a V (b A c) (left) and its corresponding BDD (right). 
The dotted lines correspond to assigning the value false to the atom whose name the edge leaves 
from. Conversely solid lines represent assignments to true. 

• Boolean functions representing the protocols for each agent, the local evaluation function for 
each agent, the valuation for the atoms; 

• Boolean functions representing the global temporal relation and the n epistemic relations 
for the agents. The Boolean formula coding the temporal relation needs to encode that 
joint actions correspond to enabled actions for all agents: /t(5;5') = V 'adjoint Act 

T AieA ai ^ Pi(h(d)), where a = (a±, . . . , a n ) is a joint action for the system and all individual 
action components aj are enabled by the local protocols at the corresponding local state h(g) 
in g. The epistemic relations for the agents can be represented simply by imposing equality 
on the corresponding local state component. 

• A Boolean formula representing the set of reachable states for the interpreted system. This 
can be encoded as standard by calculating the fix-point of the operator r(Q) = (1(g) V 
3g'(T(g,a,g')AQ(g')). 

Boolean functions are a convenient representation to perform certain logical operations on them 
(e.g., A, V); however it is well known that working out their satisfiability and validity can be 
expensive. Truth tables themselves do not offer any advantage in this respect: for instance checking 
satisfiability on them may involve checking 2 n rows of the table where n is the number of atoms 
present. OBDDs constitute a symbolic representation for Boolean functions and are normally much 
cheaper to handle. Before introducing OBDDs observe that to every Boolean function we can 
associate a binary decision tree (BDT), in which each level represents a different atom appearing in 
the Boolean function. Taking a different path along the tree corresponds to selecting a particular 
combination of values for the atoms (see Figure [1]), thereby determining the truth value of the 
formula. 

In most instances a BDT is not an efficient representation of its corresponding Boolean function. 
However, a series of operations can be performed on it to reduce it to a binary decision diagram 
(BDD). A BDD is a directed acyclic graph with an initial node, and in which each node (representing 
a Boolean atom) has two edges (corresponding to decision points true and false) originating from 
it with the final leaves being either "true" or "false" (see Figured]). There are several algorithms 
for producing BDDs from BDTs; however the order of the operations on the initial BDT affects 
the resulting BDD and, most crucially, comparing BDDs turns out to be an expensive operation. 
What makes the whole approach useful is the provable assertion that there exist sets of algorithms 
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computing canonical BDDs once the ordering of the variables is fixed. In other words, as long as 
the ordering of the variables is fixed the resulting BDD is unique for a given Boolean function. 
This is a remarkable result and leads to an alternative technique to compare Boolean functions: 
compute their canonical BDDs; if they are the same they represent the same function, if not they 
are the result of different functions. The canonical BDDs produced by this set of algorithms are 
normally referred to as OBDDs and constitute one of the leading data structures in symbolic model 
checking. We do not discuss algorithms to manipulate BDDs here and refer to [31] for details; but of 
particular significance is the fact that Boolean operations on Boolean functions can be done directly 
on the corresponding OBDDs without a very significant loss in performance. Other model-checking 
specific set operations such as computing pre-images (see below) may also be coded in terms of the 
corresponding BDDs. For more details on OBDDs and related techniques we refer to |31l Chapter 
6] and references, notably [llj . 

We now present the algorithms for the calculation of the set of states SAT(tp) satisfying a 
formula tp in C. In the OBDD approach all sets of states below are computed symbolically on the 
corresponding OBDDs. 



SAT{<p) { 

tp is an atomic formula: return {g \ V{g, tp) = true}; 

tp is -iipi: return S \ SAT(tp{)\ 

tp is ipi A tp2'- return SAT(tpi) n SAT{tpi)\ 

tp is EX<pi: return SATex(pi)', 

tp is E(^iUv? 2 ): return SAT EU (tpi,tp 2 ); 

tp is EiFtpi: return SATef^i)', 

tp is Kip\\ return SATj((tpi, i); 

tp is Ertpi: return SATe(<Pi,T); 

tp is Drtpi'- return SATr>(tpi,T); 

tp is Crtpv- return SATc(tpi,T); 

} 



In the algorithm above, the auxiliary procedures SATex,SATeu,SATef follow the standard 
algorithms used in temporal logicH For instance the set of global states satisfying EXy? is computed 
as follows (in what follows G is the set of reachable states). 

SAT E x{tp) { 
X = SAT(p); 

Y = {geG\Bg> &X and T(g,a,g')} 
return Y; 

_} 

Note that the calculation of EX involves working out the pre-image of T. The set of states 
satisfying the epistemic modalities are defined as follow (note that below we use ~p= |J igr ~« and 



SAT K (tp,i) { 


SATe(pA) { 


X = SAT(^p); 


X = SAT(^tp); 


Y = {geS\3g' eX and ~i(g,g')} 


Y = {geG\~v (g,g') and g> e X} 


return -iY; 


return -Y; 


} 


} 



2 For efficiency reasons the CTL modalities implemented are typically EX, AF, and EU. 
ACM SIGACT News 9 Vol. — No. - 



Agent SampleAgent 
Lstate = {s0,sl,s2}; 
Action = {al,a2} 
Protocol : 

sO: {al>; 

si: {a2}; 

s2: {al,a2}; 
end Protocol 
Ev: 

si if ( (AnotherAgent . Action=a7) ; 
s2 if Lstate=sl; 
end Ev 
end Agent 

Figure 2: A fragment of ISPL code describing an agent. 



SATcfaT) { 
X = SAT{ip); 
Y = G; 

while ( X ^ Y ) { 
X = Y; 

Y = {g E G | ~f (g, g') and g' E Y and g' E SAT(tp)} 
return Y; 

} 



The algorithm for Knp is similar in spirit to the CTL algorithm for computing AXip: essentially 
we compute the pre-image under the epistemic relation of the set of formulas not satisfying ip and 
negate the result. Er<p (resp., D^tp) is done similarly but on the ~^ (resp., For C we need to 

use a fix-point construction (fix-point constructions already appear in the algorithm to compute the 
satisfiability of the until operator). In fact, note that Crf = Er(y> A Cr<p), so it can be computed 
by calculating the fix-point of t(Q) = SAT(Erip A Q) as in the table above. All sets operations 
above are implemented on the corresponding OBDDs thereby producing the OBDD for SAT(<p). 
We can then solve (M, g°) \= (p by answering the query g° € SAT(<p) on the corresponding OBDD. 

3.2 MCMAS 

MCMAS [4Ql [59] is a GNU GPL tool that implements the OBDD-based procedures of the previous 
subsection. Input to the model checker is a program describing the evolutions of a multi-agent 
system. The program is given in ISPL (Interpreted Systems Programming Language), a language 
specialised for the specifications of interpreted systems and some extensions. An ISPL program 
consists of a sequence of declarations for agents in the system, valuation for the atomic propositions, 
and formulas in CTLK (other languages are also supported — see extensions). An agent is given by 
explicitly listing the local states it may be in, the local actions, protocols, and the local evolution 
function. Note that the local evolution function : Lj x Act\ x • • • x Act n — > Li gives a set of rules 
specifying the target local state when a certain combination of actions is performed. An example 
of an ISPL fragment describing a very simple agent is given in Figure [2j 

Upon invocation the tool parses the input, builds the OBDD for transition relation and the 
OBDD for the set of reachable states. This is then used in the calculation of the OBDD for the sets 
of states satisfying the formula to be verified. By comparing whether the initial state belongs to 



SAT D (^T) { 
X = SAT(^(p); 

Y = {g€G\~» (g,g') and g' E X} 
return ^Y: 

} 
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this set the output is displayed. A graphical and a web interface are available for the tool. MCMAS 
is presented in detail in [56J. 

4 SAT-based symbolic model checking 

SAT-based model checking is the most recent symbolic approach for modal logic. It was motivated 
by a dramatic increase in efficiency of SAT-solvers, i.e., algorithms solving the satisfiability problem 
for propositional formulas [68]. The main idea of SAT-based methods consists in translating the 
model checking problem for a temporal-epistemic logic to the problem of satisfiability of a formula in 
propositional logic. This formula is typically obtained by combining an encoding of the model and 
of the temporal-epistemic property. In principle, the approaches to SAT-based symbolic verification 
can be viewed as bounded (BMC) or unbounded (UMC). BMC applies to an existential fragment 
of a logic (here ECTLK) on a part of the model, whereas UMC is for an unrestricted logic (here 
CTLK) on the whole model. 

4.1 Bounded Model Checking 

BMC was originally introduced for verification of LTL (9j [8] as an alternative to approaches based 
on OBDDs. Then, BMC was defined for the existential fragment of the logic CTL [55] and then 
extended to ECTLK [53]. BMC is based on the observation that some properties of a system 
can be checked over a part of its model only. In the simplest case of reachability analysis, this 
approach consists in an iterative encoding of a finite symbolic path as a propositional formula. The 
satisfiability of the resulting propositional formula is then checked using an external SAT-solver. 
We present here the main definitions of BMC for ECTLK and later discuss extensions to more 
expressive logics. We refer the reader to the literature cited above for more details. 

To explain how the model checking problem for an ECTLK formula is encoded as a proposi- 
tional formula, we first define /c-models, bounded semantics over /c-models, and then propositional 
encodings of /c-paths in the /c-model and propositional encodings of the formulas. In order to define 
a bounded semantics for ECTLK we define k- models. Let M = (G, g°, T, ~i, . . . , ~ n , V) be a model 
and k € IN + . The /c-model for M is defined as a structure = (G, g°,Pk, ~i, • • • , V), where 
Pfc is the set of all the /c-paths of M over G, where a /c-path is the prefix of length k of a path. 

We need to identify /c-paths that represent infinite paths so that satisfaction of EG formulas 
in the bounded semantics implies their satisfaction on the unbounded one. To this aim define the 
function loop : P^ — ► 2^ as: loop(7r) = {I \ < I < k and (vr(A;), vr(/)) £ T}, which returns the 
set of indices / of 7r for which there is a transition from 7r(/c) to tt(1). 

Let Mfc be a /c-model and a, (3 be ECTLK formulas. (Mk,g) \= a denotes that a is true at the 
state g of M^. The bounded semantics is summarised as follows. {Mk,g) \= EXa has the same 
meaning as for unbounded models. (Mk,g) \= EGa states that there is a /c-path n, which starts at 
g, all its states satisfy a and tt is a loop, which means that g is a T-successor of one of the states of 
7r. The indexes of such states are given by loop(ir). For the other modalities the bounded semantics 
is the same as unbounded, insisting on reachability of the state satisfying a on a path of length k. 

Model checking over models can be reduced to model checking over /c-models. The main idea 
of BMC for ECTLK is that checking q> over is replaced by checking the satisfiability of the 
propositional formula [M, := [M^' 9 A [y]jvf fc - [M^' 9 represents (a part of) the model under 
consideration whereas [(f]M k captures a number of constraints that must be satisfied on for ip to 
be satisfied. Checking satisfiability of an ECTLK formula can be done by means of a SAT-solver. 
Typically, we start with k := 1, test satisfiability for the translation, and increase k by one until 
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either [M^' 9 A [ip]M k becomes satisfiable, or k reaches the maximal depth of M, which is bounded 
by \G\. It can be shown that if [M </3 ' 9 °]fc A [f]M k is satisfiable for some k, then (M,g°) \= cp, where 
M is the full unbounded model. 

4.1.1 Translation to SAT 

We provide here some details of the translation. The states and the transitions of the system under 
consideration are encoded similarly as for BDDs in Section [3J Let w = (w[l], . . . ,w[m]) be sequence 
of propositions (called a global state variable) for encoding global states. A sequence wqj, . . . ,Wk,j 
of global state variables is called a symbolic fc-path j. Since a model for a branching time formula 
is a tree (a set of paths), we need to use a set of symbolic /s-paths to encode it. The number of 
them depends on the value of k and the formula 99, and it is computed using the function This 
function determines the number of fc-paths sufficient for checking an ECTLK formula, see [67] for 
more details. Intuitively, each nesting of an epistemic or temporal formula in 93 increases the value 
of fk(f) by 1, whereas subformulas EU, EG and Cr add more fc-paths. 

The prepositional formula [M^' 9 representing the /c-paths in the k- model, is defined as 
follows: 

fk{<p) fc-i 

[M v,9 °]k : = I g o(wofi) A f\ f\ T(wij,w i+1 j), 

j=l i=0 

where wo t o an d Wij for < i < k and 1 < j < fk(<p) are global state variables, and T(wi y) 
is a formula encoding the transition relation T. 

An intuition behind this encoding is as follows. The vector wo t o encodes the initial state g° and 
for each symbolic fc-path, numbered 1 . . . fk(f), each pair of the consecutive vectors on this path 
encodes pairs of states that are in the transition relation T. The formula T(w, v) is typically a 
logical disjunction of the encodings of all the actions corresponding to the transitions of the model 
M. This way, one symbolic /c-path encodes all the (concrete) /c-paths. 

The next step of the algorithm consists in translating an ECTLK formula ip into a prepositional 
formula. Let w, v be global state variables. We make use of the following propositional formulas in 
the encoding: 

• p{w) encodes a proposition p of ECTLK over w. 

• H(w,v) represents logical equivalence between global state encodings u and v (i.e., encodes 
that u and v represent the same global states). 

• HKi(w, v) represents logical equivalence between idocal state encodings u and v, (i.e., encodes 
that u and v share idocal states). 

• Lk j{l) encodes a backward loop connecting the fc-th state to the l-th state in the symbolic 
fc-path j, for < I < k. 

The translation of each ECTLK formula is directly based on its bounded semantics. The translation 
of ip at the state w min into the propositional formula [^J^"'"^ is as follows (we give the translation 
of selected formulas only): 
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Intuitively, [EGa]' m,n ' fc is translated to all the /fc(y>)-symbolic fc-paths (EGa is considered as a 
subformula of cp) that start at the states encoded by w miH , satisfy a, and are loops. [if/a]^" 1 '^ is 
translated to all the /fc(</?)-symbolic fc-paths such that each symbolic /c-path starts at the initial 
state g°, one of its states satisfies a and shares the i-th state with these encoded by w m ^ n . Given 
the translations above [67], verification of p over reduces to checking the satisfiability of the 



propositional formula [M^' 9 ]j. A [</?]M fe , where [up] 



[0,0] 



4.2 Unbounded Model Checking 

UMC was originally introduced for verification of CTL |46| as an alternative to BMC and approaches 
based on BDDs. Then, UMC was extended to CTL p K [35j as well as to other more expressive logics. 

We begin by extending the syntax and semantics of CTLK to CTL p K by adding past operators 
AY and AH. The operators including Since are omitted. A backward path tt = (<7o>9i> • • •) is a 
maximal sequence of global states such that (gk+i,9k) € T for each k > (if n is finite, then k 
needs to be restricted accordingly). Let n(g) denote the set of all the backward paths starting at 

• (M,g) \= AYip iff for all tt G 11(g) we have (M,vr(l)) |= (p, 

• (M, g) \= KKip iff for all tt G U(g) and for all k > we have (M, rr(k)) \= p. 

Unlike BMC, UMC is capable of handling the whole language of the logic. Our aim is to translate 
CTLpK formulas into propositional formulas in conjunctive normal form, accepted as an input by 
SAT-solvers. 

Specifically, for a given CTL p K formula p, a corresponding propositional formula is 
computed, where w is a global state variable (i.e., a vector of propositional variables for representing 
global states) encoding these states of the model where (p holds. The translation is not operating 
directly on temporal-epistemic formulas. Instead, to calculate propositional formulas either the 
QBF or the fix-point characterisation of CTL p K formulas (see Section [3]) is used. More specifically, 
three basic algorithms are exploited. The first one, implemented by the procedure forall [M], is 
used for translating formulas 0a such that G {AX, AY, K{, Dr, Er}- This procedure eliminates 
the universal quantifiers from a QBF formula characterising a CTL p K formula, and returns the 
result in a conjunctive normal form. The second algorithm, implemented by the procedure gfpo 
is applied to formulas 0a such that G {AG, AH, Cr}. This procedure computes the greatest 
fix-point, in the standard way, using Boolean representations of sets rather than sets themselves. 
For formulas of the form A(aU/3) the third procedure, called IfpAU, computing the least fix-point 
(in a similar way), is used. In so doing, given a formula ip a propositional formula [v?](w) is obtained 
such that p is valid in the model M iff the propositional formula [p] (w) A I g o (w) is satisfiable. 

The reader is referred to [34] for more details, especially on computing fix-points over proposi- 
tional representations of sets. In the following section we show how to represent CTL p K formulas 
in QBF and then translate them to propositional formulas in CNF. 
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4.2.1 From a fragment of QBF to CNF 



Quantified Boolean Formulas (QBF) are an extension of propositional logic by means of quantifiers 
ranging over propositions. The BNF syntax of a QBF formula is given by: 

a ::= p \ ->a \ a A a \ 3p.a \ Mp.a. 
The semantics of the quantifiers is defined as follows: 

• 3p.a iff a{p <— true) V a(p <— false), 

• Mp.a iff a{p <— true) A a(p <— false), 

where a G QBF, p £ PV and a(p <— q) denotes substitution with the variable q of every occurrence 
of the variable p in formula a. For example, the formula [AXa](w) is equivalent to the formula 
Vv.(T(w,v) =4> [o](v)) in QBF. Similar equivalences are obtained for the formulas AYa, iQa, D^a, 
and Era by replacing T(w,v) with suitable encodings of the relations T" 1 , ~j, ~p , and ~p. 

For defining a translation from a fragment of QBF (resulting from the translation of CTL p K) 
to propositional logic, one needs to know how to compute a CNF formula which is equivalent to 
a given propositional formula ip. While the standard algorithm toCNF [46[ 154j . which transforms 
a propositional formula to one in CNF, preserving satisfiability only, is of linear complexity, a 
translation to an equivalent formula is NP-complete. For such a translation, one can use the 
algorithm equCNF - a version of the algorithm toCNF, known as a cube reduction. We refer the 
reader to \13\ I24j . where alternative solutions can be found. The algorithm equCNF is a slight 
modification of the DPLL algorithm checking satisfiability of a CNF formula (see [5l] ) , but it can 
be presented in a general way, abstracting away from its specific realisation. 

Assume that ip is an input formula. Initially, the algorithm equCNF builds a satisfying assign- 
ment for the formula toCNF(ip) A -iL, (/^ is a literal used in toCNF(ip)), i.e., the assignment which 
falsifies ip. If one is found, instead of terminating, the algorithm constructs a new clause that is 
in conflict with the current assignment (i.e., it rules out the satisfying assignment). Each time a 
satisfying assignment is obtained, a blocking clause is generated by a procedure blocking_clause 
and added to the working set of clauses. This clause rules out a set of cases where ip is false. 
Thus, on termination, when there is no satisfying assignment for the current set of clauses, the 
conjunction of the blocking clauses generated precisely characterises (p. 

A blocking clause could in principle be generated using the conflict-based learning procedure. 
If we require a blocking clause to contain only input variables, i.e., literals used in (p, then one 
could either use an (alternative) implication graph [16] in which all the roots are input literals 
or a method introduced by Szreter [64\ I63j . which consists in searching a directed acyclic graph 
representing the formula. 

Our aim is to compute a propositional formula equivalent to a given QBF formula Vpi . . . \/p n .<p. 
The algorithm constructs a formula ip equivalent to ip and eliminates from ip the quantified variables 
on-the-fly, which is correct as ip is in CNF. The algorithm differs from equCNF in one step only, 
where the procedure blocking_clause generates a blocking clause and deprives it of the quantified 
propositional variables. On termination, the resulting formula is a conjunction of the blocking 
clauses without the quantified propositions and precisely characterises Vpi . . . *ip n .ip (see [34t [54"] 
for more details). 
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4.3 Verics 



Verics \19\ I49j is a verification tool for real-time systems (RTS) and multi-agent systems (MAS). 
It offers three complementary methods of model checking: SAT-based Bounded Model Checking 
(BMC), SAT-based Unbounded Model Checking (UMC), and an on-the-fly verification while con- 
structing abstract models of systems. The theoretical background for its implementation has been 
presented elsewhere [Ml ES] . 

A network of communicating (timed) automata (together with a valuation function) is the basic 
Verics's formalism for modelling a system to be verified. Timed automata are used to specify RTS, 
whereas timed or untimed automata are applied to model MAS. Verics translates a network of 
automata and a temporal-epistemic formula into a propositional formula in CNF and invokes a 
SAT-solver in order to check for its satisfiability. 

Currently, Verics implements BMC for ECTLKD (ECTLK extended with deontic operators) 
and TECTLK (see Section [5]), and UMC for CTL p K. Verics has been implemented in C++; its 
internal functionalities are available via an interface written in Java [2]. 

5 Extensions to real-time epistemic logic 

In this section we briefly discuss some extensions to real-time to the ECTLK framework analysed 
so far. The timed temporal-epistemic logic TECTLK [33] was introduced to deal with situation 
where time is best assumed to be dense and hence modelled by real numbers. The underlying 
semantics uses networks of timed automata [3] to specify the behaviour of the agents. These 
automata extend standard finite state automata by a set of clocks X (to measure the flow of time) 
and time constrains built over X that can be used for defining guards on the transitions as well 
invariants on their locations. When moving from a state to another, a timed automaton can either 
execute action transitions constrained by guards and invariants, or time transitions constrained 
by invariants only. Crucial for automated verification of timed automata is the definition of an 
equivalence relation = C JR} X \ x IR'*' on clocks valuations, which identifies two valuations v and v 1 
in which either all the clocks exceed some value c max H or two clocks x and y with the same integer 
part in v and v' and either their fractional parts are equal to 0, or are ordered in the same way, i.e., 
fractional (v(x)) < fractional (v(y)) iff fractional (v'(x)) < fractional (v'(y)). The equivalence classes 
of = are called zones. Since = is of finite index, there is only finitely many zones for each timed 
automaton. 

In addition to the standard epistemic operators, the language of TECTLK contains the temporal 
operators EG and EU combined with time intervals / on reals in order to specify when precisely 
formulas are supposed to hold. Note that TECTLK does not include the next step operator EX as 
this operator is meaningless on dense time models. The formal syntax of TECTLK in BNF is as 
follows: 

ip, ip ::= p £ PV | -ip | ij) A ip | ip V <p \ K{ip \ E^(p | -DrV 9 | Cr</? | EG/^ | E^U/VO 

A (real time interpreted) model for TECTLK over a timed automaton is defined as a tuple M = 
(Q, s°,T, ~i, . . . , ~ n , V), where Q is the subset of G x IR'^' such that G is the set of locations of 
the timed automaton, all the states in Q are reachable from s° = (g°, v°) with g° being the initial 
location of the timed automaton and v° the valuation in which all the clocks are equal to 0; T is 
defined by the action and timed transitions of the timed automaton, ~j C Q x Q is an epistemic 

,! This constant is computed from a timed automaton and a formula to be verified. 
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relation for agent i defined by (g, v) ~j (</, v) iff g ~j g' and v = v'; and V : QxPV — > {true, false} 
is a valuation function for PV . Intuitively, in the above model two states are in the epistemic 
relation for agent % if their locations are in this relation according to the standard definition in 
Section [2] and their clocks valuations belong to the same zone. 

In what follows, we give the semantics of Fi(p\]j^) an d EG/y? of TECTLK and discuss how 
BMC is applied to this logic. Differently from the paths of temporal-epistemic models, the paths in 
real time models consist of action transitions interleaved with timed transitions. The time distance 
to a state s from the initial one at a given path can be computed by adding the times of all the timed 
transitions that has occurred up to this state. Following this intuition the semantics is formulated 
as follows: 

• (M, s) \= E^U/VO ^ there is a path in M starting at s which contains a state where ip holds, 
reached from s within the time distance of /, and p> holds at all the earlier states, 

• (M, s) \= EG/92 iff there is a path in M starting at s such that ip holds at all the states within 
the time distance of /. 

The idea of BMC for (M, s°) |= ip, where <p is TECTLK formula, is based on two translations and 
on the application of BMC for ECTLK. An infinite real time model M is translated to a finite 
epistemic model and each formula ip of TECTLK is translated to the formula cr(ip) of the logic 
ECTLK,,, which is a slight modification of ECTLK. The above two translations guarantee that 
(M,s°) h^iff (M d ,s°) \=cr(<p). 

Assume we are given a timed automaton A and a TECTLK formula p. We begin by translating 
the real time model M (for A) to Md- First, the automaton A is extended with one special clock y, 
an action a y , and the set of transitions E y going from each location to itself and resetting the clock 
y. These transitions are used to start the paths over which sub-formulas of ip are checked. Then, 
the finite model for the extended timed automaton is built. The model Md = (Qd, T^, ~f 

, Vd), where Qd is a suitably selected (via discretization) finite subset of Q, the relations 
T d , ~f are suitably defined restrictions of the corresponding relations in M, and Vd = V\Qd- 

The above translation cr of the temporal modalities is non-trivial only. Applying cr to E(aU//3) 
we get the formula EX ?/ E(cr(a)Ucr((/3) A p)), where the operator EX y is interpreted over the 
transitions corresponding to the action a y , and p is a propositional formula characterising zones. 
A similar translation applies to EG/a. 

After the above two translations have been defined, the model checking of a TECTLK formula 
ip over M is reduced to model checking of cr((p) over M d , for which BMC can be used as presented 
in Section UTTJ 

5.1 Example 

To exemplify the expressive power of TECTLK we specify a correctness property for an extension 
of the Railroad Crossing System (RCS) |36j, a well-known example in the literature of real time 
verification. Below, we summarise the description from |44j . 

The system consists of three agents: Train, Gate, and Controller running in parallel and syn- 
chronising through the events: approach, exit, lower and raise. When a train approaches the 
crossing, Train sends the signal approach to Controller and enters the crossing between 300 and 
500 milliseconds (ms) from this event. When Train leaves the crossing, it sends the signal exit to 
Controller. Controller sends the signal lower to Gate exactly 100ms after the signal approach is 
received, and sends the signal raise signal within 100ms after exit. Gate performs the transition 
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down within 100ms of receiving the request lower, and responds to raise by moving up between 
100ms and 200ms. 

Consider the following correctness property: there exists a behaviour of RCS such that agent 
Train considers possible a situation in which it sends the signal approach but agent Gate does not 
send the signal down within 50 ms. This property can be formalised by the following TECTLK 
formula: 

Lp = EF[ 0)OO ]ifTrom (approach A EF [0j50 ] (-.down)). 
By using BMC techniques we can verify the above property for RCS. 

6 Related Work 

The approaches above have been extended in several directions and other articles have appeared 
pursuing related lines. It was mentioned in Section 3 that van der Meyden and colleagues were the 
first to propose concretely how OBDDs could be used to model check temporal-epistemic logic as 
well as to study the complexity of the model checking problem in specific cases [47] ( }39| has further 
results on this). As discussed above the main difference of their approach to the one presented here 
is the different semantics employed and the particular optimisation techniques used on it. We refer 
to [23} [T] for more details. We are not aware of other symbolic efforts other than the one presented 
above as far as SAT-based techniques (BMC, UMC) are concerned. However, different techniques 
for temporal-epistemic logic have been put forward in the past. 

In [29] van der Hoek and Wooldridge suggested reduction of temporal-epistemic logic to temporal 
logic only by using local propositions fully describing agents' local states. The approach consists 
in manually finding appropriate propositions describing appropriate states. An example of the 
technique is described on an example in [65] where the ATL model checker MOCHA [3] is used 
(see also below). Lastly, temporal-epistemic logic on discrete time was recently recast as a special 
case of ARCTL [51] , An extension of NuSMV was introduced to implement ARCTL [51] thereby 
enabling the verification of CTLK directly on NuSMV via an ad-hoc translation as discussed in 
[38]. 

Model checking has also been investigated for certain extensions of the temporal-epistemic 
logics discussed here. In [58] an OBDD-based approach to the verification of deontic interpreted 
systems |42j is presented and in [67] the BMC case was analysed. Deontic interpreted systems are 
a formalism enabling the representation and the distinction of correct versus incorrect states of 
agents. In this framework local states are partitioned into correct and incorrect local states and 
a modality Oi introduced for every agent evaluating formulas only at the correct states thereby 
representing concepts such as "all the correct states for agent i" . For instance, one could analyse a 
variant of the dining cryptographers scenario where some cryptographers are intruders saying the 
opposite of what they should [33] . Extensions to epistemic logic to include explicit knowledge have 
also been discussed and implemented [431 141] . Both Verics and MCMAS support these formalisms. 

In other developments model checking of epistemic logic in an ATL [5] setting has also been 
pursued. ATL can be shown to extend CTL (at some computational cost) by adding strategies 
in the semantics and explicit representation of the notion of enforcement in the syntax. Even if 
strategies and knowledge can interact in rather subtle ways [32], progress has been made both in 
the definition of ATL extensions including knowledge and other modalities and in their verification. 
We refer to [66] for an up-to-date survey and references. The approach taken there uses MOCHA [3] 
and the local propositions construction referenced above. MCMAS described earlier in this survey 
also supports ATL natively in the different knowledge semantics proposed. We do not discuss the 
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syntax here and refer to above mentioned references for more details. 

Elsewhere epistemic-like concepts have been used in a broader context to reason about multi- 
agent systems modelled by other attitudes (such as norms, beliefs, desires, or intentions). Normally 
these properties are treated simply as propositions in a temporal language and not as prima-specie 
citizens like the epistemic modalities above, consequently the approaches are rather different and 
not discussed here. 



7 Conclusions 

It has long been argued that epistemic logic provides an intuitive formalism in a variety of key 
areas in computer science. In this article we have surveyed some of the recent contributions to 
solving the model checking problem for temporal-epistemic logic in a branching time setting under 
a discrete and a continuous model of time. The conclusion we can draw from the above is that 
model checking temporal-epistemic logic is very often no harder than plain temporal logic; however 
most procedures and particular algorithms need to be extended to accommodate this need. Now 
that model checking algorithms and tools have been made available it will be interesting to see the 
extent to which temporal-epistemic logic can be used in real-life scenarios. 

Note. The techniques described in Sections 3-5 were joint work of the authors with M. Kacprzak, 
F. Raimondi, and B. Wozna. 
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